Learning Natural Language: A review of formal and computational approaches

1 Overview Over the last several decades, interaction between the elds of computational learning theory and linguistics has produced a sizable body of research. The question of how people learn to understand and use language cannot be answered without rst considering what a language is, and what it means to learn something, let alone understand it. The formalization of the notion of language learning has been central to much work in inductive inference, including both algorithms for learning particular language classes under a variety of conditions and proofs that such algorithms don't exist. These theoretical investigations of learnabilty have, in turn, proven useful in linguistic theory for characterizing potential constraints on natural languages. Beyond such cross-fertilization, an independent area of research has sprung up at the intersection of the two elds to focus on the precise theoretical assumptions needed in particular linguistic frameworks to ensure that learning can take place. This paper presents a brief summary of the major work relevant to formal and computational approaches to learning natural languages. The vast majority of research so far has been directed at the acquisition of syntax as captured by formal grammars. We thus review early theoretical work on identiication of formal languages in general (Section 2) before describing subsequent work in which speciic linguistic assumptions render the problem more tractable (Section 3). A number of problems with such assumptions, however, have led to some alternative recent approaches. In particular, data-driven methods have tried to demonstrate that no language-speciic assumptions are necessary, especially in a probabilistic framework. Challenges have also risen in response to the focus on syntax and the paradigm of language learning as identiication; semantic information not only simpliies the learning problem but is also more appropriate to the non-binary nature of language use and understanding. 2 Formalities Grammatical inference as a theoretical eld of study began with Gold's (1967) attempt to formalize the acquisition of natural language. We assume the now-standard deenition of a language as the set of ((nite) strings generated by a set of production rules, or grammar, based on the symbols of a given alphabet. That is, knowledge of a language can be identiied with knowledge of the grammar that generates it, and learning a language given example strings (or sentences) is a process of inductive inference of such a grammar. The nature of the grammar determines where in the Chomsky hierarchy of complexity the language …